Simulation Experiment Recipe
Objectives
The objective of this simulation experiment is to provide a toy example on how to use simChef and showcase the automated R Markdown-generated documentation. For the sake of illustration, this toy simulation experiment studies the performance of linear regression at the surface-level with the sole purpose of facilitating an easy-to-understand walkthrough.
[Typically, the objective of the simulation experiment (and this blurb) will be more scientific than instructive and will warrant additional context/background and domain knowledge.]
Data Generation
Linear Gaussian DGP
In the Linear Gaussian DGP, we simulate the feature/design matrix \(\mathbf{X} \in \mathbb{R}^{n \times p}\) from a normal distribution and the response vector \(\mathbf{y} \in \mathbb{R}^n\) from a linear model. Specifically,
\[\begin{gather*}\mathbf{X} \sim N\left(\mathbf{0}, \begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right), \\\mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon},\\\boldsymbol{\epsilon} \sim N(\mathbf{0}, \sigma^2 \mathbf{I}_n)\end{gather*}\]
Default Parameters in DGP
- Number of samples: \(n = 200\)
- Number of features: \(p = 2\)
- Correlation among features: \(\rho = 0\)
- Amount of noise: \(\sigma = 1\)
- Coefficients: \(\boldsymbol{\beta} = (1, 0)^\top\)
[In practice, documentation of DGPs should answer the questions “what” and “why”. That is, “what” is the DGP, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
Function
#> function(n, beta, rho, sigma) {
#> cov_mat <- matrix(c(1, rho, rho, 1), byrow = T, nrow = 2, ncol = 2)
#> X <- MASS::mvrnorm(n = n, mu = rep(0, 2), Sigma = cov_mat)
#> y <- X %*% beta + rnorm(n, sd = sigma)
#> return(list(X = X, y = y))
#> }
#> <bytecode: 0x5603191d77f8>Input Parameters
#> $n
#> [1] 200
#>
#> $beta
#> [1] 1 0
#>
#> $rho
#> [1] 0
#>
#> $sigma
#> [1] 1Methods and Evaluation
Methods
OLS
Given some data \(\mathbf{X}\) and \(\mathbf{y}\), we fit ordinary least squares (OLS) and examine the p-values for each coefficient in the model. The p-values are computed using a two-sided t-test (see summary.lm()).
Elaborating further on the testing, we are interested in testing:
\[\begin{align*}H_0: \beta_i = 0 \quad vs. \quad H_1: \beta_i \neq 0.\end{align*}\]
To test this, we compute the observed T-statistic, defined as
\[\begin{align*}T = \frac{\hat{\beta}_i}{\hat{SE}(\hat{\beta_i})},\end{align*}\]
and then compute the two-sided p-value under the t distribution with \(n - p - 1\) degrees of freedom. If the p-value is lower than some significance value \(\alpha\), then there is sufficient evidence to reject the null hypothesis \(H_0\). Otherwise, there is not sufficient evidence, and we fail to reject the null hypothesis.
[In practice, documentation of methods should answer the questions “what” and “why”. That is, “what” is the method, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
Function
#> function(X, y, cols = c("X1", "X2")) {
#> lm_fit <- lm(y ~ X)
#> pvals <- summary(lm_fit)$coefficients[cols, "Pr(>|t|)"] %>%
#> setNames(paste(names(.), "p-value"))
#> return(pvals)
#> }
#> <bytecode: 0x56031bdb1908>Input Parameters
#> list()Evaluation
Rejection Prob. (alpha = 0.1)
We define the rejection probability as the proportion of repetitions in the simulation experiment that result in a p-value \(\leq \alpha\). Here, we choose to set the significance level \(\alpha = 0.1\).
[In practice, documentation of evaluation metrics should answer the questions “what” and “why”. That is, “what” is the metric, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
Base
Function
#> function(fit_results, alpha = 0.05) {
#> group_vars <- c(".dgp_name", ".method_name")
#> eval_out <- fit_results %>%
#> dplyr::group_by(across({{group_vars}})) %>%
#> dplyr::summarise(
#> `X1 Reject Prob.` = mean(`X1 p-value` < alpha),
#> `X2 Reject Prob.` = mean(`X2 p-value` < alpha)
#> )
#> return(eval_out)
#> }Input Parameters
#> $alpha
#> [1] 0.1Linear Gaussian DGP - Varying beta
Function
#> function(fit_results, vary_params = NULL, alpha = 0.05) {
#> group_vars <- c(".dgp_name", ".method_name", vary_params)
#> eval_out <- fit_results %>%
#> dplyr::group_by(across({{group_vars}})) %>%
#> dplyr::summarise(
#> `X1 Reject Prob.` = mean(`X1 p-value` < alpha),
#> `X2 Reject Prob.` = mean(`X2 p-value` < alpha)
#> )
#> return(eval_out)
#> }
#> <bytecode: 0x56031ac28060>Input Parameters
#> $alpha
#> [1] 0.1Linear Gaussian DGP - Varying rho
Function
#> function(fit_results, vary_params = NULL, alpha = 0.05) {
#> group_vars <- c(".dgp_name", ".method_name", vary_params)
#> eval_out <- fit_results %>%
#> dplyr::group_by(across({{group_vars}})) %>%
#> dplyr::summarise(
#> `X1 Reject Prob.` = mean(`X1 p-value` < alpha),
#> `X2 Reject Prob.` = mean(`X2 p-value` < alpha)
#> )
#> return(eval_out)
#> }
#> <bytecode: 0x56031bbcecc8>Input Parameters
#> $alpha
#> [1] 0.1Linear Gaussian DGP - Varying sigma
Function
#> function(fit_results, vary_params = NULL, alpha = 0.05) {
#> group_vars <- c(".dgp_name", ".method_name", vary_params)
#> eval_out <- fit_results %>%
#> dplyr::group_by(across({{group_vars}})) %>%
#> dplyr::summarise(
#> `X1 Reject Prob.` = mean(`X1 p-value` < alpha),
#> `X2 Reject Prob.` = mean(`X2 p-value` < alpha)
#> )
#> return(eval_out)
#> }Input Parameters
#> $alpha
#> [1] 0.1Visualizations
Power
To examine the power of the test, we plot the rejection probability as a function of \(\alpha\), that is, \(\mathbb{P}(\text{p-value} \leq \alpha)\) vs. \(\alpha\). If the coefficient is non-zero in the underlying DGP, then a larger AUC would indicate better performance in terms of the power. We will primarily focus on plotting the power of the first coefficient \(\beta_1\).
[In practice, documentation of the plotters should answer the questions “what” and “why”. That is, “what” is the plot, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
Base
Function
#> function(fit_results, col = "X1") {
#> plt <- ggplot2::ggplot(fit_results) +
#> ggplot2::aes(x = .data[[paste(col, "p-value")]],
#> color = as.factor(.method_name)) +
#> ggplot2::geom_abline(slope = 1, intercept = 0,
#> color = "darkgray", linetype = "solid", size = 1) +
#> ggplot2::stat_ecdf(size = 1) +
#> ggplot2::scale_x_continuous(limits = c(0, 1)) +
#> ggplot2::labs(x = "t", y = "P( p-value \u2264 t )",
#> linetype = "", color = "Method")
#> return(plt)
#> }Input Parameters
#> list()Linear Gaussian DGP - Varying beta
Function
#> function(fit_results, vary_params = NULL, col = "X1") {
#>
#> if (!is.null(vary_params)) {
#> # deal with the case when we vary across a parameter that is vector-valued
#> if (is.list(fit_results[[vary_params]])) {
#> fit_results[[vary_params]] <- list_col_to_chr(fit_results[[vary_params]],
#> name = vary_params,
#> verbatim = TRUE)
#> }
#> }
#>
#> plt <- ggplot2::ggplot(fit_results) +
#> ggplot2::aes(x = .data[[paste(col, "p-value")]],
#> color = as.factor(.method_name)) +
#> ggplot2::geom_abline(slope = 1, intercept = 0,
#> color = "darkgray", linetype = "solid", size = 1) +
#> ggplot2::stat_ecdf(size = 1) +
#> ggplot2::scale_x_continuous(limits = c(0, 1)) +
#> ggplot2::labs(x = "t", y = "P( p-value \u2264 t )",
#> linetype = "", color = "Method")
#> if (!is.null(vary_params)) {
#> plt <- plt + ggplot2::facet_wrap(~ .data[[vary_params]])
#> }
#> return(plt)
#> }
#> <bytecode: 0x560315970b18>Input Parameters
#> list()Linear Gaussian DGP - Varying rho
Function
#> function(fit_results, vary_params = NULL, col = "X1") {
#>
#> if (!is.null(vary_params)) {
#> # deal with the case when we vary across a parameter that is vector-valued
#> if (is.list(fit_results[[vary_params]])) {
#> fit_results[[vary_params]] <- list_col_to_chr(fit_results[[vary_params]],
#> name = vary_params,
#> verbatim = TRUE)
#> }
#> }
#>
#> plt <- ggplot2::ggplot(fit_results) +
#> ggplot2::aes(x = .data[[paste(col, "p-value")]],
#> color = as.factor(.method_name)) +
#> ggplot2::geom_abline(slope = 1, intercept = 0,
#> color = "darkgray", linetype = "solid", size = 1) +
#> ggplot2::stat_ecdf(size = 1) +
#> ggplot2::scale_x_continuous(limits = c(0, 1)) +
#> ggplot2::labs(x = "t", y = "P( p-value \u2264 t )",
#> linetype = "", color = "Method")
#> if (!is.null(vary_params)) {
#> plt <- plt + ggplot2::facet_wrap(~ .data[[vary_params]])
#> }
#> return(plt)
#> }
#> <bytecode: 0x56031505b8d8>Input Parameters
#> list()Linear Gaussian DGP - Varying sigma
Function
#> function(fit_results, vary_params = NULL, col = "X1") {
#>
#> if (!is.null(vary_params)) {
#> # deal with the case when we vary across a parameter that is vector-valued
#> if (is.list(fit_results[[vary_params]])) {
#> fit_results[[vary_params]] <- list_col_to_chr(fit_results[[vary_params]],
#> name = vary_params,
#> verbatim = TRUE)
#> }
#> }
#>
#> plt <- ggplot2::ggplot(fit_results) +
#> ggplot2::aes(x = .data[[paste(col, "p-value")]],
#> color = as.factor(.method_name)) +
#> ggplot2::geom_abline(slope = 1, intercept = 0,
#> color = "darkgray", linetype = "solid", size = 1) +
#> ggplot2::stat_ecdf(size = 1) +
#> ggplot2::scale_x_continuous(limits = c(0, 1)) +
#> ggplot2::labs(x = "t", y = "P( p-value \u2264 t )",
#> linetype = "", color = "Method")
#> if (!is.null(vary_params)) {
#> plt <- plt + ggplot2::facet_wrap(~ .data[[vary_params]])
#> }
#> return(plt)
#> }
#> <bytecode: 0x560314b76628>Input Parameters
#> list()Rejection Prob. (alpha = 0.1) Plot
We plot the rejection probability for \(\beta_1\) across varying parameters of the DGP to understand how characteristics of the DGP affect the test.We define the rejection probability as the proportion of repetitions in the simulation experiment that result in a p-value \(\leq \alpha\). Here, we choose to set the significance level \(\alpha = 0.1\).
[In practice, documentation of the plotters should answer the questions “what” and “why”. That is, “what” is the plot, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
Linear Gaussian DGP - Varying beta
Function
#> function(eval_results, vary_params = NULL,
#> alpha = 0.05) {
#> eval_results <- eval_results$`Rejection Prob. (alpha = 0.1)`
#> if (is.list(eval_results[[vary_params]])) {
#> # deal with the case when we vary across a parameter that is vector-valued
#> eval_results[[vary_params]] <- list_col_to_chr(eval_results[[vary_params]],
#> name = vary_params,
#> verbatim = TRUE)
#> }
#> plt <- ggplot2::ggplot(eval_results) +
#> ggplot2::aes(x = .data[[vary_params]], y = `X1 Reject Prob.`,
#> color = as.factor(.method_name),
#> fill = as.factor(.method_name)) +
#> ggplot2::labs(x = vary_params,
#> y = sprintf("Rejection Probability (alpha = %s)", alpha),
#> color = "Method", fill = "Method") +
#> ggplot2::scale_y_continuous(limits = c(0, 1))
#> if (is.numeric(eval_results[[vary_params]])) {
#> plt <- plt +
#> ggplot2::geom_line() +
#> ggplot2::geom_point(size = 2)
#> } else {
#> plt <- plt +
#> ggplot2::geom_bar(stat = "identity")
#> }
#> return(plotly::ggplotly(plt))
#> }
#> <bytecode: 0x5603158bf550>Input Parameters
#> $alpha
#> [1] 0.1Linear Gaussian DGP - Varying rho
Function
#> function(eval_results, vary_params = NULL,
#> alpha = 0.05) {
#> eval_results <- eval_results$`Rejection Prob. (alpha = 0.1)`
#> if (is.list(eval_results[[vary_params]])) {
#> # deal with the case when we vary across a parameter that is vector-valued
#> eval_results[[vary_params]] <- list_col_to_chr(eval_results[[vary_params]],
#> name = vary_params,
#> verbatim = TRUE)
#> }
#> plt <- ggplot2::ggplot(eval_results) +
#> ggplot2::aes(x = .data[[vary_params]], y = `X1 Reject Prob.`,
#> color = as.factor(.method_name),
#> fill = as.factor(.method_name)) +
#> ggplot2::labs(x = vary_params,
#> y = sprintf("Rejection Probability (alpha = %s)", alpha),
#> color = "Method", fill = "Method") +
#> ggplot2::scale_y_continuous(limits = c(0, 1))
#> if (is.numeric(eval_results[[vary_params]])) {
#> plt <- plt +
#> ggplot2::geom_line() +
#> ggplot2::geom_point(size = 2)
#> } else {
#> plt <- plt +
#> ggplot2::geom_bar(stat = "identity")
#> }
#> return(plotly::ggplotly(plt))
#> }
#> <bytecode: 0x560314f28790>Input Parameters
#> $alpha
#> [1] 0.1Base Linear Regression Experiment
Rejection Prob. (alpha = 0.1)
Power
Linear Gaussian DGP
Varying beta
Rejection Prob. (alpha = 0.1)
Power
Rejection Prob. (alpha = 0.1) Plot
Parameter Values
#> $dgp
#> $dgp$`Linear Gaussian DGP`
#> $dgp$`Linear Gaussian DGP`$beta
#> $dgp$`Linear Gaussian DGP`$beta[[1]]
#> [1] 1 0
#>
#> $dgp$`Linear Gaussian DGP`$beta[[2]]
#> [1] 1.0 0.5
#>
#> $dgp$`Linear Gaussian DGP`$beta[[3]]
#> [1] 1 1
#>
#> $dgp$`Linear Gaussian DGP`$beta[[4]]
#> [1] 1.0 1.5
#>
#> $dgp$`Linear Gaussian DGP`$beta[[5]]
#> [1] 1 2
#>
#>
#>
#>
#> $method
#> list()Varying rho
Rejection Prob. (alpha = 0.1)
Power
Rejection Prob. (alpha = 0.1) Plot
Parameter Values
#> $dgp
#> $dgp$`Linear Gaussian DGP`
#> $dgp$`Linear Gaussian DGP`$rho
#> [1] 0.0 0.2 0.5 0.9
#>
#>
#>
#> $method
#> list()Varying sigma
Rejection Prob. (alpha = 0.1)
Power
Parameter Values
#> $dgp
#> $dgp$`Linear Gaussian DGP`
#> $dgp$`Linear Gaussian DGP`$sigma
#> [1] 1 2 4 8
#>
#>
#>
#> $method
#> list()LS0tCnRpdGxlOiAiYHIgcGFyYW1zJHNpbV9uYW1lYCIKYXV0aG9yOiAiYHIgcGFyYW1zJGF1dGhvcmAiCmRhdGU6ICJgciBmb3JtYXQoU3lzLnRpbWUoKSwgJyVCICVkLCAlWScpYCIKaGVhZGVyLWluY2x1ZGVzOgogICAgLSBcdXNlcGFja2FnZXtmbG9hdH0KICAgIC0gXHVzZXBhY2thZ2V7YW1zbWF0aH0KICAgIC0gXHVzZXBhY2thZ2V7Z2Vuc3ltYn0Kb3V0cHV0OgogIHZ0aGVtZXM6OnZtb2Rlcm46CiAgICB0aHVtYm5haWxzOiBmYWxzZQpjc3M6IGNzcy9zaW1jaGVmLmNzcwpwYXJhbXM6CiAgYXV0aG9yOiAKICAgIGxhYmVsOiAiQXV0aG9yOiIKICAgIHZhbHVlOiAiIgogIHNpbV9uYW1lOgogICAgbGFiZWw6ICJTaW11bGF0aW9uIEV4cGVyaW1lbnQgTmFtZToiCiAgICB2YWx1ZTogIiIKICBzaW1fcGF0aDoKICAgIGxhYmVsOiAiUGF0aCB0byBTaW11bGF0aW9uIEV4cGVyaW1lbnQgRm9sZGVyOiIKICAgIHZhbHVlOiAiIgogIGV2YWxfb3JkZXI6CiAgICBsYWJlbDogIk9yZGVyIG9mIEV2YWx1YXRvcnM6IgogICAgdmFsdWU6IE5VTEwKICB2aXpfb3JkZXI6CiAgICBsYWJlbDogIk9yZGVyIG9mIFZpc3VhbGl6ZXJzOiIKICAgIHZhbHVlOiBOVUxMCiAgdmVyYm9zZToKICAgIGxhYmVsOiAiVmVyYm9zZSBMZXZlbDoiCiAgICB2YWx1ZTogMgotLS0KCjxzY3JpcHQgc3JjPSJqcy9zaW1jaGVmTmF2Q2xhc3MuanMiPjwvc2NyaXB0PgoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9Cm9wdGlvbnMod2lkdGggPSAxMDAwMCkKa25pdHI6Om9wdHNfY2h1bmskc2V0KAogIGVjaG8gPSBGQUxTRSwKICB3YXJuaW5nID0gRkFMU0UsCiAgbWVzc2FnZSA9IEZBTFNFLAogIGNhY2hlID0gRkFMU0UsCiAgZmlnLmFsaWduID0gImNlbnRlciIsCiAgZmlnLnBvcyA9ICJIIiwKICBmaWcuaGVpZ2h0ID0gMTIsCiAgZmlnLndpZHRoID0gMTAKKQoKb3B0aW9ucyhrbml0ci5rYWJsZS5OQSA9ICdOQScsCiAgICAgICAgZHBseXIuc3VtbWFyaXNlLmluZm9ybSA9IEZBTFNFKQoKIyBzY3JvbGxhYmxlIHRleHQgb3V0cHV0CmxvY2FsKHsKICBob29rX291dHB1dCA8LSBrbml0cjo6a25pdF9ob29rcyRnZXQoJ291dHB1dCcpCiAga25pdHI6OmtuaXRfaG9va3Mkc2V0KG91dHB1dCA9IGZ1bmN0aW9uKHgsIG9wdGlvbnMpIHsKICAgIGlmICghaXMubnVsbChvcHRpb25zJG1heC5oZWlnaHQpKSBvcHRpb25zJGF0dHIub3V0cHV0IDwtIGMoCiAgICAgIG9wdGlvbnMkYXR0ci5vdXRwdXQsCiAgICAgIHNwcmludGYoJ3N0eWxlPSJtYXgtaGVpZ2h0OiAlczsiJywgb3B0aW9ucyRtYXguaGVpZ2h0KQogICAgKQogICAgaG9va19vdXRwdXQoeCwgb3B0aW9ucykKICB9KQp9KQoKY2h1bmtfaWR4IDwtIDEKZG9jX2RpciA8LSBmaWxlLnBhdGgocGFyYW1zJHNpbV9wYXRoLCAiZG9jcyIpCmBgYAoKYGBge3IgaGVscGVyLWZ1bnN9CgojJyBHZXQgb3JkZXIgb2Ygb2JqZWN0cyB0byBkaXNwbGF5CiMnCiMnIEBwYXJhbSBvYmpfbmFtZXMgVmVjdG9yIG9mIGFsbCBvYmplY3QgbmFtZXMgdGhhdCBuZWVkIHRvIGJlIGRpc3BsYXllZC4KIycgQHBhcmFtIG9ial9vcmRlciBWZWN0b3Igb2Ygb2JqZWN0IG5hbWVzIGluIHRoZSBkZXNpcmVkIGFwcGVhcmFuY2Ugb3JkZXIuCiMnIEByZXR1cm4gVmVjdG9yIG9mIG9iamVjdCBuYW1lcyBpbiB0aGUgb3JkZXIgaW4gd2hpY2ggdGhleSB3aWxsIGJlIGRpc3BsYXllZC4KZ2V0T2JqT3JkZXIgPC0gZnVuY3Rpb24ob2JqX25hbWVzLCBvYmpfb3JkZXIgPSBOVUxMKSB7CiAgaWYgKGlzLm51bGwob2JqX29yZGVyKSkgewogICAgcmV0dXJuKG9ial9uYW1lcykKICB9IGVsc2UgewogICAgcmV0dXJuKGludGVyc2VjdChvYmpfb3JkZXIsIG9ial9uYW1lcykpCiAgfQp9CgojJyBHZXQgYWxsIGV4cGVyaW1lbnRzIHVuZGVyIGEgZ2l2ZW4gZGlyZWN0b3J5IG5hbWUKIycKIycgQHBhcmFtIGRpcl9uYW1lIG5hbWUgb2YgZGlyZWN0b3J5CiMnIEByZXR1cm4gbGlzdCBvZiBuYW1lZCBleHBlcmltZW50cwpnZXREZXNjZW5kYW50cyA8LSBmdW5jdGlvbihkaXJfbmFtZSkgewogIGV4cGVyaW1lbnRzIDwtIGxpc3QoKQogIGZvciAoZCBpbiBsaXN0LmRpcnMoZGlyX25hbWUpKSB7CiAgICBpZiAoZmlsZS5leGlzdHMoZmlsZS5wYXRoKGQsICJleHBlcmltZW50LnJkcyIpKSkgewogICAgICBpZiAoaWRlbnRpY2FsKGQsIHBhcmFtcyRzaW1fcGF0aCkpIHsKICAgICAgICBleHBfbmFtZSA8LSAiQmFzZSIKICAgICAgfSBlbHNlIHsKICAgICAgICBleHBfbmFtZSA8LSBzdHJpbmdyOjpzdHJfcmVwbGFjZV9hbGwoCiAgICAgICAgICBzdHJpbmdyOjpzdHJfcmVtb3ZlKGQsIHBhc3RlMChwYXJhbXMkc2ltX3BhdGgsICIvIikpLAogICAgICAgICAgIi8iLCAiIC0gIgogICAgICAgICkKICAgICAgfQogICAgICBleHBlcmltZW50c1tbZXhwX25hbWVdXSA8LSByZWFkUkRTKGZpbGUucGF0aChkLCAiZXhwZXJpbWVudC5yZHMiKSkKICAgIH0KICB9CiAgcmV0dXJuKGV4cGVyaW1lbnRzKQp9CgojJyBDaGVjayBpZiBleHBlcmltZW50IGV4aXN0cwojJwojJyBAcGFyYW0gZGlyX25hbWUgbmFtZSBvZiBkaXJlY3Rvcnkgb3IgdmVjdG9yIHRoZXJlb2YKIycgQHBhcmFtIHJlY3Vyc2l2ZSBsb2dpY2FsOyBpZiBUUlVFLCBjaGVja3MgaWYgZXhwZXJpbWVudCBleGlzdHMgdW5kZXIgdGhlCiMnICAgZ2l2ZW4gZGlyZWN0b3J5KHMpOyBpZiBGQUxTRSwgY2hlY2tzIGlmIGFueSBleHBlcmltZW50IGV4aXN0cyB1bmRlciB0aGUKIycgICBkaXJlY3RvcnkocykgYW5kIGl0cyBkZXNjZW5kYW50cwojJyBAcmV0dXJuIFRSVUUgaWYgZXhwZXJpbWVudCBleGlzdHMgYW5kIEZBTFNFIG90aGVyd2lzZQpleHBlcmltZW50RXhpc3RzIDwtIGZ1bmN0aW9uKGRpcl9uYW1lLCByZWN1cnNpdmUgPSBGQUxTRSkgewogIHJlcyA8LSBwdXJycjo6bWFwX2xnbChkaXJfbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgZnVuY3Rpb24oZCkgewogICAgICAgICAgICAgICAgICAgICAgICAgIGlmICghcmVjdXJzaXZlKSB7CiAgICAgICAgICAgICAgICAgICAgICAgICAgICBleHBfZm5hbWUgPC0gZmlsZS5wYXRoKGQsICJleHBlcmltZW50LnJkcyIpCiAgICAgICAgICAgICAgICAgICAgICAgICAgICByZXR1cm4oZmlsZS5leGlzdHMoZXhwX2ZuYW1lKSkKICAgICAgICAgICAgICAgICAgICAgICAgICB9IGVsc2UgewogICAgICAgICAgICAgICAgICAgICAgICAgICAgZGVzY2VuZGFudHMgPC0gZ2V0RGVzY2VuZGFudHMoZCkKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJldHVybihsZW5ndGgoZGVzY2VuZGFudHMpID4gMCkKICAgICAgICAgICAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgICAgICAgICAgICAgIH0pCiAgcmV0dXJuKGFueShyZXMpKQp9CgojJyBEaXNwbGF5cyBjb250ZW50IGZvciBzcGVjaWZpZWQgcGFydCBvZiByZWNpcGUKIycKIycgQHBhcmFtIGZpZWxkX25hbWUgcGFydCBvZiByZWNpcGUgdG8gc2hvdzsgbXVzdCBiZSBvbmUgb2YgImRncCIsICJtZXRob2QiLAojJyAgICJldmFsdWF0b3IiLCBvciAidmlzdWFsaXplciIKIycgQHJldHVybiBjb250ZW50IGZvciByZWNpcGUKc2hvd1JlY2lwZVBhcnQgPC0gZnVuY3Rpb24oZmllbGRfbmFtZSA9IGMoImRncCIsICJtZXRob2QiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiZXZhbHVhdG9yIiwgInZpc3VhbGl6ZXIiKSkgewoKICBmaWVsZF9uYW1lIDwtIG1hdGNoLmFyZyhmaWVsZF9uYW1lKQogIGZ1bmNfbmFtZSA8LSBkcGx5cjo6Y2FzZV93aGVuKGZpZWxkX25hbWUgPT0gImV2YWx1YXRvciIgfiAiZXZhbCIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmllbGRfbmFtZSA9PSAidmlzdWFsaXplciIgfiAidml6IiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBUUlVFIH4gZmllbGRfbmFtZSkKICBkZXNjZW5kYW50cyA8LSBnZXREZXNjZW5kYW50cyhkaXJfbmFtZSA9IHBhcmFtcyRzaW1fcGF0aCkKICBvYmpzIDwtIHB1cnJyOjptYXAoZGVzY2VuZGFudHMsIH4ueFtbcGFzdGUwKCJnZXRfIiwgZmllbGRfbmFtZSwgInMiKV1dKCkpCiAgb2JqX25hbWVzIDwtIHVuaXF1ZShwdXJycjo6cmVkdWNlKHNhcHBseShvYmpzLCBuYW1lcyksIGMpKQoKICBvYmpfaGVhZGVyIDwtICI8cCBzdHlsZT0nZm9udC13ZWlnaHQ6IGJvbGQ7IGZvbnQtc2l6ZTogMjBweCc+ICVzIDwvcD4iCiAgaW52aXNfaGVhZGVyIDwtICJcblxuIyMjICVzIHsudGFic2V0IC50YWJzZXQtcGlsbHMgLnRhYnNldC1yZWNpcGUgLnRhYnNldC1jaXJjbGV9XG5cbiIKICBzaG93dHlwZV9oZWFkZXIgPC0gIlxuXG4jIyMjICVzIHsudGFic2V0IC50YWJzZXQtcGlsbHN9XG5cbiIKICBleHBfaGVhZGVyIDwtICJcblxuIyMjIyMgJXMgXG5cbiIKCiAgaWYgKGFsbChzYXBwbHkob2JqcywgbGVuZ3RoKSA9PSAwKSkgewogICAgcmV0dXJuKGNhdCgiTi9BIikpCiAgfQoKICBmb3IgKGlkeCBpbiAxOmxlbmd0aChvYmpfbmFtZXMpKSB7CiAgICBjYXQoc3ByaW50ZihpbnZpc19oZWFkZXIsICIiKSkKICAgIG9ial9uYW1lIDwtIG9ial9uYW1lc1tpZHhdCgogICAgY2F0KCI8ZGl2IGNsYXNzPSdwYW5lbCBwYW5lbC1kZWZhdWx0IHBhZGRlZC1wYW5lbCc+IikKICAgIGNhdChzcHJpbnRmKG9ial9oZWFkZXIsIG9ial9uYW1lKSkKCiAgICBjYXQoc3ByaW50ZihzaG93dHlwZV9oZWFkZXIsIGZvbnRhd2Vzb21lOjpmYSgicmVhZG1lIiwgZmlsbCA9ICJ3aGl0ZSIpKSkKICAgIHBhc3RlTWQoZmlsZS5wYXRoKGRvY19kaXIsIHBhc3RlMChmaWVsZF9uYW1lLCAicyIpLAogICAgICAgICAgICAgICAgICAgICAgcGFzdGUwKG9ial9uYW1lLCAiLm1kIikpKQoKICAgIGNhdChzcHJpbnRmKHNob3d0eXBlX2hlYWRlciwgZm9udGF3ZXNvbWU6OmZhKCJjb2RlIiwgZmlsbCA9ICJ3aGl0ZSIpKSkKICAgIGtlZXBfb2JqcyA8LSBwdXJycjo6bWFwKG9ianMsIG9ial9uYW1lKQogICAga2VlcF9vYmpzW3NhcHBseShrZWVwX29ianMsIGlzLm51bGwpXSA8LSBOVUxMCiAgICBpZiAoYWxsKHB1cnJyOjptYXBfbGdsKGtlZXBfb2JqcywKICAgICAgICAgICAgICAgICAgICAgICAgICAgfmlzVFJVRShjaGVja19lcXVhbCgueCwga2VlcF9vYmpzW1sxXV0pKSkpKSB7CiAgICAgIG9iaiA8LSBrZWVwX29ianNbWzFdXQogICAgICBjYXQoIjxiPkZ1bmN0aW9uPC9iPiIpCiAgICAgIHZ0aGVtZXM6OnN1YmNodW5raWZ5KG9ialtbcGFzdGUwKGZ1bmNfbmFtZSwgIl9mdW4iKV1dLAogICAgICAgICAgICAgICAgICBjaHVua19pZHgsIG90aGVyX2FyZ3MgPSAibWF4LmhlaWdodD0nMjAwcHgnIikKICAgICAgY2h1bmtfaWR4IDw8LSBjaHVua19pZHggKyAxCiAgICAgIGNhdCgiPGI+SW5wdXQgUGFyYW1ldGVyczwvYj4iKQogICAgICB2dGhlbWVzOjpzdWJjaHVua2lmeShvYmpbW3Bhc3RlMChmdW5jX25hbWUsICJfcGFyYW1zIildXSwKICAgICAgICAgICAgICAgICAgY2h1bmtfaWR4LCBvdGhlcl9hcmdzID0gIm1heC5oZWlnaHQ9JzIwMHB4JyIpCiAgICAgIGNodW5rX2lkeCA8PC0gY2h1bmtfaWR4ICsgMQogICAgfSBlbHNlIHsKICAgICAgZm9yIChleHAgaW4gbmFtZXMob2JqcykpIHsKICAgICAgICBvYmogPC0gb2Jqc1tbZXhwXV1bW29ial9uYW1lXV0KICAgICAgICBpZiAoaXMubnVsbChvYmopKSB7CiAgICAgICAgICBuZXh0CiAgICAgICAgfQogICAgICAgIGNhdChzcHJpbnRmKGV4cF9oZWFkZXIsIGV4cCkpCiAgICAgICAgY2F0KCI8Yj5GdW5jdGlvbjwvYj4iKQogICAgICAgIHZ0aGVtZXM6OnN1YmNodW5raWZ5KG9ialtbcGFzdGUwKGZ1bmNfbmFtZSwgIl9mdW4iKV1dLAogICAgICAgICAgICAgICAgICAgIGNodW5rX2lkeCwgb3RoZXJfYXJncyA9ICJtYXguaGVpZ2h0PScyMDBweCciKQogICAgICAgIGNodW5rX2lkeCA8PC0gY2h1bmtfaWR4ICsgMQogICAgICAgIGNhdCgiPGI+SW5wdXQgUGFyYW1ldGVyczwvYj4iKQogICAgICAgIHZ0aGVtZXM6OnN1YmNodW5raWZ5KG9ialtbcGFzdGUwKGZ1bmNfbmFtZSwgIl9wYXJhbXMiKV1dLAogICAgICAgICAgICAgICAgICAgIGNodW5rX2lkeCwgb3RoZXJfYXJncyA9ICJtYXguaGVpZ2h0PScyMDBweCciKQogICAgICAgIGNodW5rX2lkeCA8PC0gY2h1bmtfaWR4ICsgMQogICAgICB9CiAgICB9CiAgICBjYXQoIjwvZGl2PiIpCiAgfQp9CgojJyBSZWFkcyBpbiBmaWxlIGlmIGl0IGV4aXN0cyBhbmQgcmV0dXJucyBOVUxMIGlmIHRoZSBmaWxlIGRvZXMgbm90IGV4aXN0CiMnCiMnIEBwYXJhbSBmaWxlbmFtZSBuYW1lIG9mIC5yZHMgZmlsZSB0byB0cnkgcmVhZGluZyBpbgojJyBAcmV0dXJuIG91dHB1dCBvZiBmaWxlbmFtZS5yZHMgaWYgdGhlIGZpbGUgZXhpc3RzIGFuZCBOVUxMIG90aGVyd2lzZQpnZXRSZXN1bHRzIDwtIGZ1bmN0aW9uKGZpbGVuYW1lKSB7CiAgaWYgKGZpbGUuZXhpc3RzKGZpbGVuYW1lKSkgewogICAgcmVzdWx0cyA8LSByZWFkUkRTKGZpbGVuYW1lKQogIH0gZWxzZSB7CiAgICByZXN1bHRzIDwtIE5VTEwKICB9CiAgcmV0dXJuKHJlc3VsdHMpCn0KCiMnIERpc3BsYXlzIG91dHB1dCAoYm90aCBmcm9tIGV2YWx1YXRlKCkgYW5kIHZpc3VhbGl6ZSgpKSBmcm9tIHNhdmVkIHJlc3VsdHMgdW5kZXIKIycgYSBzcGVjaWZpZWQgZGlyZWN0b3J5CiMnCiMnIEBwYXJhbSBkaXJfbmFtZSBuYW1lIG9mIGRpcmVjdG9yeQojJyBAcGFyYW0gZGVwdGggaW50ZWdlcjsgZGVwdGggb2YgZGlyZWN0b3J5IGZyb20gcGFyZW50L2Jhc2UgZXhwZXJpbWVudCdzIGZvbGRlcgojJyBAcGFyYW0gYmFzZSBsb2dpY2FsOyB3aGV0aGVyIG9yIG5vdCB0aGlzIGlzIGEgYmFzZSBleHBlcmltZW50CiMnIEBwYXJhbSBzaG93X2hlYWRlciBsb2dpY2FsOyB3aGV0aGVyIG9yIG5vdCB0byBzaG93IHNlY3Rpb24gaGVhZGVyCiMnIEBwYXJhbSB2ZXJib3NlIGludGVnZXI7IDAgPSBubyBtZXNzYWdlczsgMSA9IHByaW50IG91dCBkaXJlY3RvcnkgbmFtZSBvbmx5OwojJyAgIDIgPSBwcmludCBvdXQgZGlyZWN0b3J5IG5hbWUgYW5kIG5hbWUgb2YgZXZhbHVhdG9ycy92aXN1YWxpemVycwojJyBAcmV0dXJuIGNvbnRlbnQgcmVzdWx0cyBmcm9tIGV2YWx1YXRlKCkgYW5kIHZpc3VhbGl6ZSgpIGZyb20gdGhlIGV4cGVyaW1lbnQKc2hvd1Jlc3VsdHMgPC0gZnVuY3Rpb24oZGlyX25hbWUsIGRlcHRoLCBiYXNlID0gRkFMU0UsIHNob3dfaGVhZGVyID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZSA9IDEpIHsKICBpZiAodmVyYm9zZSA+PSAxKSB7CiAgICBtZXNzYWdlKHJlcCgiKiIsIGRlcHRoKSwgYmFzZW5hbWUoZGlyX25hbWUpKQogIH0KCiAgaWYgKGRlcHRoID09IDEpIHsKICAgIGhlYWRlcl90ZW1wbGF0ZSA8LSAiXG5cbiVzICVzIHsudGFic2V0IC50YWJzZXQtcGlsbHMgLnRhYnNldC12bW9kZXJufVxuXG4iCiAgfSBlbHNlIHsKICAgIGlmIChiYXNlIHwgIWV4cGVyaW1lbnRFeGlzdHMoZGlyX25hbWUpKSB7CiAgICAgIGhlYWRlcl90ZW1wbGF0ZSA8LSAiXG5cbiVzICVzIHsudGFic2V0IC50YWJzZXQtcGlsbHN9IgogICAgfSBlbHNlIHsKICAgICAgaGVhZGVyX3RlbXBsYXRlIDwtICJcblxuJXMgJXMgey50YWJzZXQgLnRhYnNldC1waWxscyAudGFic2V0LWNpcmNsZX0iCiAgICB9CiAgfQoKICBpZiAoc2hvd19oZWFkZXIpIHsKICAgIGNhdChzcHJpbnRmKGhlYWRlcl90ZW1wbGF0ZSwKICAgICAgICAgICAgICAgIHBhc3RlKHJlcCgiIyIsIGRlcHRoKSwgY29sbGFwc2UgPSAiIiksCiAgICAgICAgICAgICAgICBiYXNlbmFtZShkaXJfbmFtZSkpKQogIH0KCiAgaWYgKGJhc2UpIHsKICAgIGNhdChwYXN0ZTAoIlxuXG4iLAogICAgICAgICAgICAgICBwYXN0ZShyZXAoIiMiLCBkZXB0aCArIDEpLCBjb2xsYXBzZSA9ICIiKSwKICAgICAgICAgICAgICAgIiBCYXNlIC0gIiwgYmFzZW5hbWUoZGlyX25hbWUpLAogICAgICAgICAgICAgICAiIHsudGFic2V0IC50YWJzZXQtcGlsbHMgLnRhYnNldC1jaXJjbGV9XG5cbiIpKQogICAgZGVwdGggPC0gZGVwdGggKyAxCiAgfQoKICBzaG93dHlwZV90ZW1wbGF0ZSA8LSBwYXN0ZTAoCiAgICAiXG5cbiIsIHBhc3RlKHJlcCgiIyIsIGRlcHRoICsgMSksIGNvbGxhcHNlID0gIiIpLCAiICVzXG5cbiIKICApCiAgZmlnbmFtZV90ZW1wbGF0ZSA8LSAiPGgzIHN0eWxlPSdmb250LXdlaWdodDogYm9sZCc+ICVzIDwvaDM+IgogIGludmlzaWJsZV9oZWFkZXIgPC0gcGFzdGUwKAogICAgIlxuXG4iLCBwYXN0ZShyZXAoIiMiLCBkZXB0aCArIDIpLCBjb2xsYXBzZSA9ICIiKSwKICAgICIgey50YWJzZXQgLnRhYnNldC1waWxsc31cblxuIgogICkKICBwbHRfdGVtcGxhdGUgPC0gcGFzdGUwKAogICAgIlxuXG4iLCBwYXN0ZShyZXAoIiMiLCBkZXB0aCArIDMpLCBjb2xsYXBzZSA9ICIiKSwgIiAlc1xuXG4iCiAgKQoKICBleHBfZm5hbWUgPC0gZmlsZS5wYXRoKGRpcl9uYW1lLCAiZXhwZXJpbWVudC5yZHMiKQogICMgZml0X2ZuYW1lIDwtIGZpbGUucGF0aChkaXJfbmFtZSwgImZpdF9yZXN1bHRzLnJkcyIpCiAgZXZhbF9mbmFtZSA8LSBmaWxlLnBhdGgoZGlyX25hbWUsICJldmFsX3Jlc3VsdHMucmRzIikKICB2aXpfZm5hbWUgPC0gZmlsZS5wYXRoKGRpcl9uYW1lLCAidml6X3Jlc3VsdHMucmRzIikKCiAgZXhwIDwtIGdldFJlc3VsdHMoZXhwX2ZuYW1lKQogICMgZml0X3Jlc3VsdHMgPC0gZ2V0UmVzdWx0cyhmaXRfZm5hbWUpCiAgZXZhbF9yZXN1bHRzIDwtIGdldFJlc3VsdHMoZXZhbF9mbmFtZSkKICB2aXpfcmVzdWx0cyA8LSBnZXRSZXN1bHRzKHZpel9mbmFtZSkKCiAgaWYgKCFpcy5udWxsKGV2YWxfcmVzdWx0cykpIHsKICAgIGNhdChzcHJpbnRmKHNob3d0eXBlX3RlbXBsYXRlLCBmb250YXdlc29tZTo6ZmEoInRhYmxlIiwgZmlsbCA9ICJ3aGl0ZSIpKSkKICAgIGV2YWxfbmFtZXMgPC0gZ2V0T2JqT3JkZXIobmFtZXMoZXZhbF9yZXN1bHRzKSwgcGFyYW1zJGV2YWxfb3JkZXIpCiAgICBmb3IgKGV2YWxfbmFtZSBpbiBldmFsX25hbWVzKSB7CiAgICAgIGlmICh2ZXJib3NlID49IDIpIHsKICAgICAgICBtZXNzYWdlKHJlcCgiICIsIGRlcHRoICsgMSksIGV2YWxfbmFtZSkKICAgICAgfQogICAgICBldmFsdWF0b3IgPC0gZXhwJGdldF9ldmFsdWF0b3JzKClbW2V2YWxfbmFtZV1dCiAgICAgIGlmIChldmFsdWF0b3IkZG9jX3Nob3cpIHsKICAgICAgICBjYXQoc3ByaW50ZihmaWduYW1lX3RlbXBsYXRlLCBldmFsX25hbWUpKQogICAgICAgIGRvLmNhbGwodnRoZW1lczo6cHJldHR5X0RULAogICAgICAgICAgICAgICAgYyhsaXN0KGV2YWxfcmVzdWx0c1tbZXZhbF9uYW1lXV0pLCBldmFsdWF0b3IkZG9jX29wdGlvbnMpKSAlPiUKICAgICAgICAgIHZ0aGVtZXM6OnN1YmNodW5raWZ5KGkgPSBjaHVua19pZHgpCiAgICAgICAgY2h1bmtfaWR4IDw8LSBjaHVua19pZHggKyAxCiAgICAgIH0KICAgIH0KICB9CgogIGlmICghaXMubnVsbCh2aXpfcmVzdWx0cykpIHsKICAgIGNhdChzcHJpbnRmKHNob3d0eXBlX3RlbXBsYXRlLAogICAgICAgICAgICAgICAgZm9udGF3ZXNvbWU6OmZhKCJjaGFydC1iYXIiLCBmaWxsID0gIndoaXRlIikpKQogICAgdml6X25hbWVzIDwtIGdldE9iak9yZGVyKG5hbWVzKHZpel9yZXN1bHRzKSwgcGFyYW1zJHZpel9vcmRlcikKICAgIGZvciAodml6X25hbWUgaW4gdml6X25hbWVzKSB7CiAgICAgIGlmICh2ZXJib3NlID49IDIpIHsKICAgICAgICBtZXNzYWdlKHJlcCgiICIsIGRlcHRoICsgMSksIHZpel9uYW1lKQogICAgICB9CiAgICAgIHZpc3VhbGl6ZXIgPC0gZXhwJGdldF92aXN1YWxpemVycygpW1t2aXpfbmFtZV1dCiAgICAgIGlmICh2aXN1YWxpemVyJGRvY19zaG93KSB7CiAgICAgICAgY2F0KGludmlzaWJsZV9oZWFkZXIpCiAgICAgICAgY2F0KHNwcmludGYoZmlnbmFtZV90ZW1wbGF0ZSwgdml6X25hbWUpKQogICAgICAgIHBsdHMgPC0gdml6X3Jlc3VsdHNbW3Zpel9uYW1lXV0KICAgICAgICBpZiAoIWluaGVyaXRzKHBsdHMsICJsaXN0IikpIHsKICAgICAgICAgIHBsdHMgPC0gbGlzdChwbHQgPSBwbHRzKQogICAgICAgIH0KICAgICAgICBpZiAoaXMubnVsbChuYW1lcyhwbHRzKSkpIHsKICAgICAgICAgIG5hbWVzKHBsdHMpIDwtIDE6bGVuZ3RoKHBsdHMpCiAgICAgICAgfQogICAgICAgIGZvciAocGx0X25hbWUgaW4gbmFtZXMocGx0cykpIHsKICAgICAgICAgIGlmIChsZW5ndGgocGx0cykgIT0gMSkgewogICAgICAgICAgICBjYXQoc3ByaW50ZihwbHRfdGVtcGxhdGUsIHBsdF9uYW1lKSkKICAgICAgICAgIH0KICAgICAgICAgIHBsdCA8LSBwbHRzW1twbHRfbmFtZV1dCiAgICAgICAgICBpZiAoaW5oZXJpdHMocGx0LCAicGxvdGx5IikgfCBpbmhlcml0cyhwbHQsICJnZyIpIHwgaW5oZXJpdHMocGx0LCAiZ2dwbG90IikpIHsKICAgICAgICAgICAgYWRkX2NsYXNzIDwtIGMoInBhbmVsIHBhbmVsLWRlZmF1bHQgcGFkZGVkLXBhbmVsIikKICAgICAgICAgIH0gZWxzZSB7CiAgICAgICAgICAgIGFkZF9jbGFzcyA8LSBOVUxMCiAgICAgICAgICB9CiAgICAgICAgICB2dGhlbWVzOjpzdWJjaHVua2lmeShwbHQsIGkgPSBjaHVua19pZHgsCiAgICAgICAgICAgICAgICAgICAgICBmaWdfaGVpZ2h0ID0gdmlzdWFsaXplciRkb2Nfb3B0aW9ucyRoZWlnaHQsCiAgICAgICAgICAgICAgICAgICAgICBmaWdfd2lkdGggPSB2aXN1YWxpemVyJGRvY19vcHRpb25zJHdpZHRoLAogICAgICAgICAgICAgICAgICAgICAgb3RoZXJfYXJncyA9ICJvdXQud2lkdGggPSAnMTAwJSciLAogICAgICAgICAgICAgICAgICAgICAgYWRkX2NsYXNzID0gYWRkX2NsYXNzKQogICAgICAgICAgY2h1bmtfaWR4IDw8LSBjaHVua19pZHggKyAxCiAgICAgICAgfQogICAgICB9CiAgICB9CiAgfQoKICBpZiAoIWlzLm51bGwoZXhwKSkgewogICAgaWYgKChsZW5ndGgoZXhwJGdldF92YXJ5X2Fjcm9zcygpJGRncCkgIT0gMCkgfAogICAgICAgIChsZW5ndGgoZXhwJGdldF92YXJ5X2Fjcm9zcygpJG1ldGhvZCkgIT0gMCkpIHsKICAgICAgY2F0KHNwcmludGYoc2hvd3R5cGVfdGVtcGxhdGUsIGZvbnRhd2Vzb21lOjpmYSgiY29kZSIsIGZpbGwgPSAid2hpdGUiKSkpCiAgICAgIGNhdCgiPGI+UGFyYW1ldGVyIFZhbHVlczwvYj4iKQogICAgICB2dGhlbWVzOjpzdWJjaHVua2lmeShleHAkZ2V0X3ZhcnlfYWNyb3NzKCksCiAgICAgICAgICAgICAgICAgIGNodW5rX2lkeCwgb3RoZXJfYXJncyA9ICJtYXguaGVpZ2h0PScyMDBweCciKQogICAgICBjaHVua19pZHggPDwtIGNodW5rX2lkeCArIDEKICAgIH0KICB9Cn0KCiMnIERpc3BsYXlzIG91dHB1dCBvZiBleHBlcmltZW50IGZvciBhbGwgb2YgaXRzIChzYXZlZCkgZGVzY2VuZGFudHMKIycKIycgQHBhcmFtIGRpcl9uYW1lIG5hbWUgb2YgcGFyZW50IGV4cGVyaW1lbnQgZGlyZWN0b3J5CiMnIEBwYXJhbSBkZXB0aCBwbGFjZWhvbGRlciBmb3IgcmVjdXJzaW9uOyBzaG91bGQgbm90IGJlIG1lc3NlZCB3aXRoCiMnIEBwYXJhbSAuLi4gb3RoZXIgYXJndW1lbnRzIHRvIHBhc3MgaW50byBzaG93UmVzdWx0cygpCnNob3dEZXNjZW5kYW50UmVzdWx0cyA8LSBmdW5jdGlvbihkaXJfbmFtZSwgZGVwdGggPSAxLCAuLi4pIHsKICBjaGlsZHJlbiA8LSBsaXN0LmRpcnMoZGlyX25hbWUsIHJlY3Vyc2l2ZSA9IEZBTFNFKQogIGlmIChsZW5ndGgoY2hpbGRyZW4pID09IDApIHsKICAgIHJldHVybigpCiAgfQogIGZvciAoY2hpbGRfaWR4IGluIDE6bGVuZ3RoKGNoaWxkcmVuKSkgewogICAgY2hpbGQgPC0gY2hpbGRyZW5bY2hpbGRfaWR4XQogICAgaWYgKCFleHBlcmltZW50RXhpc3RzKGNoaWxkLCByZWN1cnNpdmUgPSBUUlVFKSkgewogICAgICBuZXh0CiAgICB9CiAgICBpZiAoZXhwZXJpbWVudEV4aXN0cyhjaGlsZCwgcmVjdXJzaXZlID0gRkFMU0UpICYKICAgICAgICAoZXhwZXJpbWVudEV4aXN0cyhsaXN0LmRpcnMoY2hpbGQsIHJlY3Vyc2l2ZSA9IFRSVUUpWy0xXSkgfAogICAgICAgICAoZGVwdGggPT0gMSkpKSB7CiAgICAgIGJhc2UgPC0gVFJVRQogICAgfSBlbHNlIHsKICAgICAgYmFzZSA8LSBGQUxTRQogICAgfQogICAgc2hvd1Jlc3VsdHMoY2hpbGQsIGRlcHRoLCBiYXNlID0gYmFzZSwgLi4uKQogICAgc2hvd0Rlc2NlbmRhbnRSZXN1bHRzKGNoaWxkLCBkZXB0aCArIDEsIC4uLikKICB9Cn0KCgpgYGAKCiMgU2ltdWxhdGlvbiBFeHBlcmltZW50IFJlY2lwZSB7LnRhYnNldCAudGFic2V0LXZtb2Rlcm59CgojIyBPYmplY3RpdmVzIHsucGFuZWwgLnBhbmVsLWRlZmF1bHQgLnBhZGRlZC1wYW5lbH0KCmBgYHtyIG9iamVjdGl2ZXMsIHJlc3VsdHMgPSAiYXNpcyJ9CnBhc3RlTWQoZmlsZS5wYXRoKGRvY19kaXIsICJvYmplY3RpdmVzLm1kIikpCmBgYAoKIyMgRGF0YSBHZW5lcmF0aW9uCgpgYGB7ciBkZ3BzLCByZXN1bHRzID0gImFzaXMifQpzaG93UmVjaXBlUGFydChmaWVsZF9uYW1lID0gImRncCIpCmBgYAoKIyMgTWV0aG9kcyBhbmQgRXZhbHVhdGlvbgoKIyMjIE1ldGhvZHMKCmBgYHtyIG1ldGhvZHMsIHJlc3VsdHMgPSAiYXNpcyJ9CnNob3dSZWNpcGVQYXJ0KGZpZWxkX25hbWUgPSAibWV0aG9kIikKYGBgCgojIyMgRXZhbHVhdGlvbgoKYGBge3IgZXZhbHVhdG9ycywgcmVzdWx0cyA9ICJhc2lzIn0Kc2hvd1JlY2lwZVBhcnQoZmllbGRfbmFtZSA9ICJldmFsdWF0b3IiKQpgYGAKCiMjIFZpc3VhbGl6YXRpb25zCgpgYGB7ciB2aXN1YWxpemVycywgcmVzdWx0cyA9ICJhc2lzIn0Kc2hvd1JlY2lwZVBhcnQoZmllbGRfbmFtZSA9ICJ2aXN1YWxpemVyIikKYGBgCgoKCmBgYHtyIHJlcywgcmVzdWx0cyA9ICJhc2lzIn0KCiMgc2hvdyByZXN1bHRzCmlmIChleHBlcmltZW50RXhpc3RzKHBhcmFtcyRzaW1fcGF0aCkpIHsKICBjYXQoc3ByaW50ZigiXG5cbiMgQmFzZSAlcyBcblxuIiwgcGFyYW1zJHNpbV9uYW1lKSkKICBjYXQoIlxuXG4jIyB7LnRhYnNldCAudGFic2V0LXBpbGxzIC50YWJzZXQtY2lyY2xlfVxuXG4iKQogIG1lc3NhZ2Uoc3ByaW50ZigiQ3JlYXRpbmcgUiBNYXJrZG93biByZXBvcnQgZm9yICVzLi4uIiwgcGFyYW1zJHNpbV9uYW1lKSkKICBzaG93UmVzdWx0cyhwYXJhbXMkc2ltX3BhdGgsIGRlcHRoID0gMiwgYmFzZSA9IEZBTFNFLCBzaG93X2hlYWRlciA9IEZBTFNFLAogICAgICAgICAgICAgIHZlcmJvc2UgPSAwKQp9CgpzaG93RGVzY2VuZGFudFJlc3VsdHMocGFyYW1zJHNpbV9wYXRoLCB2ZXJib3NlID0gcGFyYW1zJHZlcmJvc2UpCgpgYGAK